The Mouthesizer : A Facial Gesture Musical Interface
نویسنده
چکیده
Introduction In this technical sketch we introduce a new interaction paradigm: using facial gestures and expressions to control musical sound. The mouthesizer, a prototype of this general concept, consists of a miniature head mounted camera which acquires video input from the region of the mouth, extracts the mouth shape using a simple computer vision algorithm, and converts shape parameters to MIDI commands to control a synthesizer or musical effects device.
منابع مشابه
Designing, Playing, and Performing with a Vision-based Mouth Interface
The role of the face and mouth in speech production as well as non-verbal communication suggests the use of facial action to control musical sound. Here we document work on the Mouthesizer, a system which uses a headworn miniature camera and computer vision algorithm to extract shape parameters from the mouth opening and output these as MIDI control changes. We report our experience with variou...
متن کاملQuantative Evaluation of the Efficiency of Facial Bio-potential Signals Based on Forehead Three-Channel Electrode Placement For Facial Gesture Recognition Applicable in a Human-Machine Interface
Introduction: Today, facial bio-potential signals are employed in many human-machine interface applications for enhancing and empowering the rehabilitation process. The main point to achieve that goal is to record appropriate bioelectric signals from the human face by placing and configuring electrodes over it in the right way. In this paper, heuristic geometrical position and configuration of ...
متن کاملThe Case Study of Application of Advanced Gesture Interface and Mapping Interface, - Virtual Musical Instrument "Le SuperPolm" and Gesture Controller "BodySuit"
We will discuss the case study of application of the Virtual Musical Instrument and Sound Synthesis. Doing this application, the main subject is advanced Mapping Interface in order to connect these. For this experiment, our discussion also refers to Neural Network, as well as a brief introduction of the Virtual Musical Instrument “Le SuperPolm” and Gesture Controller “BodySuit”.
متن کاملSonification of Facial Actions for Musical Expression
The central role of the face in social interaction and non-verbal communication suggest we explore facial action as a means of musical expression. This paper presents the design, implementation, and preliminary studies of a novel system utilizing face detection and optic flow algorithms to associate facial movements with sound synthesis in a topographically specific fashion. We report on our ex...
متن کاملGesture and Musical Interaction: Interactive Engagement Through Dynamic Morphology
New Interfaces for Musical Expression must speak to the nature of ‘instrument’, that is, it must always be understood that the interface binds to a complex musical phenomenon. This paper explores the nature of engagement, the point of performance that occurs when a human being engages with a computer based instrument. It asks questions about the nature of the instrument in computer music and of...
متن کامل